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Related Appeals and Interferences: 

None 

Status of Claims: 

Claims 1-25 are pending. 

Claim 26 is withdrawn from consideration. 

Claims 1-25 stand rejected under 35 U.S.C. § 103(a) as being unpatentable over 
Tognazzini et al. (5,886,683) in view of Jones (OB 2,170,910). 

Status of Amendments: 

Amendment "After Final", filed 6719/2003, not entered. 
Summary of the Invention; 

The present invention provides fox a system and method for recognizing, from eye-gaze 
patterns, when the user is reading, skimming, or scanning on a display rilled with heterogeneous 
content Heterogeneous content includes objects normally encountered on computer monitors, 
such as text, images, hyperlinks, windows, icons, and menus. In one embodiment, the system 
uses information about what text the user is wading or skimming to infer user interest and uses 
this information to adapt to the user's needs. The adaptation process includes recording the text 
of interest in a user model and using the text to find related information from local machine 
^tahacpo local area network databases, or wide area network databases such as the World Wide 
Web. 

The present invention's method for recognizing, from eye-gaze patterns, when a user is 
reading, skimming, or scanning, comprises at least the steps of: (1) quantized representation of 
eye-movements of a user viewing heterogeneous content (e.g. text, images, hyperlinks, windows, 
icons, menus, etc.), (2) accumulated evidence based detection, and (3) mode switching. First, the 
eye-movements in both x and y positions are quantized (e.g., averaged), in one embodiment over 
100 ms intervals. This process removes some of the inaccuracy of prior art eye-tracking 
hardware and reduces the influence of micro-saccades. Second, evidence of reading is 
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accumulated until it crosses a threshold value. The system may increment a reading-evidence 
variable by 1, for instance, when the eye moves to the right and de-incrementing by 1, for 
insto.ee, when the eye moves to the left. If the rrndiny~rvider.ee reaches a threshold, then 
••reading" is detected and the mode switched ton scanning to reading. If the threshold is not 
reached, then the system continues to collect evidence of reading. 



Pending Claims (all citations are made from the original specification, including the figures): 

1 . A method for recognizing reading, skimming, and scanning modes' from eye-gaze 
patterns (Title, Abstract, Snnuna^ of invention, page 7, line. 12-13, etc), said method 

comprising the steps of: 

quantizing eye movements of a user (figure 6, element 602) viewing heterogeneous 
content (page 9, line 11) in both X andV axes(page 9, line 15)-, accumulating a numerical 
evidence of reading (figure 6, element 604) until a predeBrrnined threshold is reached (figure 
6, element 60S); and 

detecting reading when said numerical evidence of reading exceeds said threshold (figure 

6, element 610). 

2. A method for recognizing reading, skimming, and scanning modes from eye-gaze 
patterns, as per claim 1 , wherein said quantizing step includes averaging over a predetermined 
period of time (page 9, line 15). 

3. A method for recognizing reading, slamming, and scanning modes from eye-gaze 
patterns, as per claim 1 , wherein said heterogeneous content includes at least one of the 
following: text, images, hyperlinks, windows, icons, and menus (Summary of Invention, page 

7, lines 14,15). 

4. A method for recognizing reading, skimming, and scanning modes from eye-gaze 
patterns, as per claim 1 , comprising the further step of switching modes from a skimming mode 
and a scanning mode to a reading mode when said reading is detected (figures 5-7, elements 
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506, 612, 710, respectively). 



5. A method for recognizing reading, skimming, and scanning modes from eye-gaze 
patterns, as per claim 4, comprising the further step of adapting to said user's need if said reading 
is detected (figure 9, 906). 

6. A method for recognizing reading, stimung. and scanning modes from eye-gaze 
patterns, as per claim 5, wherein said adapting step comprises the further steps of: 

recording said heterogeneous content that is of interest to said user (figure 10, 1002); 

finding relevant information from a database (figure 10, 1004) using said recorded 
heterogeneous content; and utilizing said found relevant information to adapt to a user's 
needs(figurel0,1006). 

7. A method for recognizing reading, skimming, and scanning modes from eye-gaze 
patterns, as per claim 6, wherein said utilizing step comprises the further step of developing 
accurate models of said users (figure 8, 804). 

8. A method for recognizing reading, skimming, and scanning modes from eye-gaze 
patterns, as per claim 6, wherein said utilizing step comprises the further step of determining 
fine-grained information regarding said user's interest in Internet advertising (figure 8, 806). 

9. A method for recognizing reading, slamming, and scanning modes from eye-gaze 
patterns, as per claim 6, wherein said utilizing step comprises the further step of providing more 
accurate help regarding computer applications (figure 8, 808). 

10. A method for recognizing reading, slamming, and scanning modes from eye-gaze 
patterns, as per claim 6, wherein said database includes at least one of the following: a local 
machine database (figure 11, 1102), a local area network database (figure 11, 1104), and a wide 
area network database (figure 11, 1106). 
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U . A method for recognizing reading, skimming, and scanning modes from eye-gaze 
patterns, as per claim 1 0, wherein said wide area network database is the world wide web (page 
13, line 19). 

12. A system for recognizing reading, skimming, and scanning modes from eye-gaze patterns 
(figure 7, page 10, starting on tine 1$), said system comprising: 

an eye-movement quantizer (figure 7, 702 )that quantizes eye movements of a user 
viewing heterogeneous content in both X and Y axes; 

a reading-evidence accumulator (figure 7, 704) that accumulates a numerical evidence of 

reading; 

a threshold-comparator (figure 7, 706 )that compares said numerical evidence of reading 
against a predetermined threshold; and 

a reading-detector (figure 7, 708 )that detects reading when said numerical evidence of 
reading exceeds said predetermined threshold. 

1 3. A system for recognizing reading, skimming, and scanning modes from eye-gaze 
patterns, as per claim 12, wherein said eye-movement quantizer averages over a predetermined 
period of time (page 9, line IS). 

14. A system for recognizing reading, skimming, and scanning modes from eye-gaze 
patterns, as per claim 12, wherein said heterogeneous content includes at least one of the 
following: text, images, hyperlinks, windows, icons, and menus (Summary of Invention, page 
7, lines 14,15). 

1 5. A system for recognizing reading, skimming, and scanning modes from eye-gaze 
patterns, as per claim 12, further comprising a mode-switcher (figure 7, 710) that switches 
modes from a skimming mode and a scanning mode to a reading mode when said reading is 
detected. 

16. A system for recognizing reading, skimming, and scanning modes from eye-gaze 
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patens, as per claim 1 5, wherein said modc-switcher generates 8 user adaptation command 
signal when switching to said reading mode (figure 9, 906). 

17. A system for recognizing reading, skimming, and scanning modes from eye-gaze 
patterns, as per claim 16, wherein, in response to said user adaptation command signal: 

records said heterogeneous content that is of interest to said user (figure 10, 1002); 

finds relevant information fiom a database (figure 10, 1004) using said recorded 
heterogeneous content; and utilizes said found relevant information to adapt to a user's 
needs(figurel0,!006). 

18. A system for recognizing reading, skimming, and scanning modes fiom eye-gaze 
patterns, as per claim 1 7, wherein the system develops accurate models of said users (figure 8, 
804). 

19. A system for recognizing reading, skimming, and scanning modes from eye-gaze 
patterns, as per claim 17, wherein the system determines fine-grained information regarding said 
user's interest in Internet advertising (figure 8, 806). 

20. A system for recognizing reading, skimming, and scanning modes from eye-gaze 
patterns, as per claim 17, wherein the system provides more accurate help regarding computer 
applications (figure 8, 808). 

21. A system for recognizing reading, skimming, and scanning modes fiom eye-gaze 
patterns, as per claim 17, wherein said database includes at least one of the following: a local 
machine database(figure 11, 1102), a local area network database (figure 11, 1104), and a wide 
area network database (figure 11, 1106). 



22. A system for recognizing reading, skimming, and scanning modes from eye-gaze 
patterns, as per claim 21, wherein said wide area network database is the world wide web (page 
13, line 19). 
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23. A system for recognizing reading, skimming, and scanning modes from eye-gaze patterns 
(figure 7, page 10, starting on line 16), said system comprising: 

means for quantizing eye movements (figure 7, 702) of a user viewing heterogeneous 
content in both X and Y axes; 

means for accumulating a numerical evidence of reading (figure 7, 704); 
means for comparing said numerical evidence of reading against a predetermined 

threshold (figure 7, 706); and 

means for detecting reading when said numerical evidence of reading exceeds said 
predetermined threshold (figure 7, 708). 

24. An article of manufacture comprising a computer program product having a machine- 
readable medium including computer program instructions embodied therein for recogiuzing 
reading, skirnniing, and scanning ftom eye-gaze patterns (figure 7, page 10, starting on line 16) 

with: 

computer program instructions for quantizing eye movements of a user viewing 
heterogeneous content in both X and Y axes (figure 7, 702), said quantizing based on averaging 
over a predetermined period of time (page 9, line 15); 

computer program instructions for accumulating a numerical evidence of reading until a 
predetermined threshold is reached (figure 7, 704); 

computer program instructions for detecting reading when said numerical evidence of 
leading exceeds said predetermined threshold (figure 7, 706); and 

computer program instructions for switching modes from a scanning mode and a 
coming mode to a leading mode when reading is detected (figure 7, 710). 

25. An article of manufacture comprising a computer program product having a machine- 
readable medium including computer program instructions embodied therein for utilizing user 
interest information to adapt a computer to a user's needs (figure 9, 900) with: 

computer program instructions for recording eye-gaze patterns of said user viewing 
heterogeneous content (figure 9, 902); 
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computer program instructions for determining from said recorded patterns whether said 
user is reading, skimming, and scanning (figure 9, 904); 

computer program instructions for recording heterogeneous content of interest to said 
user upon detection of said reading (figure 10, 1002); 

computer program instructions for finding relevant information from a database using 
said recorded heterogeneous content of interest (figure 10, 1004); and 

computer program instructions fox adapting said computer to said user's needs using said 
relevant information (figure 10, 1006). 

Issues: 

I. Was the interview "After Final" process conducted in good feith and according to proper 

USPTO procedures? 

II. Were the reasons for non-entry of the amendment "After Final" proper? 

HI. Does the combination of references (Tognazzim et al. (5,886,683) in view of Jones (GB 
2, 170,91 0)) provide for the elements of the claims of the present invention? 

IV. Has the examiner established a prima facia case of obviousness for the outstanding 
rejection of clams 1-25 under 35 U.S.C § 103(a) as being unpatentable over Tognazzini 
et al, (5,886,683) in view of Jones (GB 2,170,910)? 

Grouping of Claims: 

All claims stand or fall together. 

Argument 

I. An "Interview after Final" was granted and conducted on 7/17*2003, with follow-up 
interviews on 7/17/03 and 8/7/03. The interviews were initially with examiner Chanh 
Nguyen and applicant's representatives Randy Lacasse and Ramraj Sourjdararajan. The 
interview of 8/7/03 was with Randy Lacasse and the examiner's supervisor Steven Saras. 
During the first interview, the exarniner agreed that the applied references were different 
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than the claimed inventioiL However, he felt that some additional language would assist 
in more clearly defining this relationship. The language was chosen, upon mutual 
agreement, from the argument put forth in applicant's first amendment Specifically, the 
language was chosen ftom the entered amendment of record of 2/7/03, page 4, lines 1-5 
and page 6, lines 7-13. The examiner was very helpful in determining this language, and 
indicated that the claims as amended by this language would fully overcome the 
rejections under Togoazzini and Jones (see examiner's interview summary dated 
6717/03). Applicants' representative accordingly submitted such as an amendment "After 
Final" (as fully discussed in the interview). 
H The examiner then submitted an "Advisory action" indicating that the amendment 
presented new issues not earlier presented and would require a new search. 

a. Applicants' representatives were surprised by the examiner's change in position 
from the interview and contacted him to discuss. Hisanswer was contrary to 
traditional USPTO practice. He indicated that he had conducted the interview 
"After Final" only to develop claim language which would be allowable upon the 
filing ofanRCE. At no time during the interview had the examiner discussed 
filing an RCE to advance the prosecution of the case. We acted in accordance 
with his specific guidance to make the amendments as discussed and feel that he 
acted in bad faith by holding the allowance in abeyance in exchange for 
applicant's filing of an RCE. 

b. In a subsequent interview with the examiner's SPE, these tacts were discussed 
and it was indicated to us (by the SPE) that the examiner should not have 
requested for us to file an RCE. However, the SPE reaffirmed that the 
amendment "After Final" would not be entered because it raised new issues which 
required an additional search. 

The applicant's position, with respect to the non-entry of the amendment after final is: 
1 . Acknowledgment that neither the interview nor subsequent amendment are by matter 

of right, but rather by examiner discretion. We appreciated the opportunity to 

interview and submit the amendment "After Final". 
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2. However, the examiner appears to have acted in bad faith by agreeing to specific 
claim language to overcome the references of record and then requiring an RCE as 
passageway for entry. 

3. The examiner cited that the amendment after final language raised new issues, yet this 
language was presented during an earlier amendment (arguments) and thus was of 
record and known to the examiner during prosecution. Also, the language was part of 
the original specification (page 9, lines 18+). 

REJECTIONS UNDER 35 U .S.C. $ 103(a) 

The Examiner has rejected claims 1-25 under U.S.C § 103(a) as being unpatentable over 
Tognazzini in view of Jones. With respect to these claims (claims 1-25), applicants respectfully 
disagree with the Examiner as they contend that the presently claimed invention cannot be 
obvious in view of any of the references because, even if there were a suggestion to combine 
them, a combination of these references would not result in the presently claimed invention. 

The U.S. patent to Tognazziiii et at. (hereon Tognazzini) appears to disclose an effort to 
capitalize on eye-gaze as a measure of user interest Tognazzini describes a method and 
apparatus for providing relevant information based on eye-gaze. For example, interest in some 
display object is deternuned based on a fixation. If the user looks at an object on the screen long 
enough, the system infers that the user is interested in that object. This same rule also applies to 
blocks of text. Throughout Tognazzini, all formulations are based on this time of gaze aspect 
(fixation). This differs greatly from the presently claimed invention as will be shown hereafter in 
more detail. 

With respect to claims 1, 12, 23, 24, and 25, the examiner states that Tognazzini discloses 
a method of recognizing reading, skimming, and scanning modes from eye gaze patterns On col. 
4, lines 50-54). The first examiner citation appears to be in error as column 4, lines 50-54, 
simply discus well known traditional newspaper layouts that use a technique of moving parts of 
articles to different pages to enable quick reading. This does not describe a "method for 
recognizing reading, skimming, and scanning modes from eye gaze patterns (claim 1), or a 
"system" to perform this function (claims 12 and 23), or an article of manufacture comprising a 
computer program product (claims 24 and 25). There is no basis for the examiner's assertion 



10 



Serial No. 09/649,608 
Group Art Unit 2675 
Docket No: ARC920000027US1 

with respect to the column 4 reading. 

The examiner further stales that Tognazzini discloses a method of recognizing reading, 
skimming, and scanning modes from eye gaze patterns (in col. 17, lines 1-5). A closer reading 
of Tognazzini, however, indicates that the disclosed method determines whether the user 
skimmed the herniation or read the information by "measuring the amount of time spent with 
each article and advertisement" (see column 17, lines 1-5 of the tognazzini reference). First, 
Tognazzini does not recognize (or suggest) a scanning mode. Second, Tognazzini detects 
regions of interest (not actual reading) by determining the time spent gazing at a particular region 
of the computer screen (figure 4, figure 1 3, elements 1305, 1311). 

The examiner asserts mat Tognazzini mils to mention the limitation of "accumulating a 
numerical evidence of rending until a predetennined threshold is reached; and detecting reading 
when the numerical evidence of reading exceeds the threshold." These are very important 
features of the claims. Features the examiner has admitted that me main reference does not 
provide or suggest. The suggestion, teaching, or motivation to combine a secondary reference 
with this main reference to overcome the lack of claimed elements must come from within the 
references themselves, not from a reconstruction* of the invention based on applicant's own 
teachings. It is applicant's position that nothing within the references would lead one to mate 
this combination, nor would the aimbination provide for the claimed invention (even if properly 
combined). 

The Foreign patent (GB 2170910) to Jones has been used by the examiner to support a 
finding of obviousness based on a combination with the main reference Tognazzini. Jones 
appears to provide for an eye tracking system with a control signal that is responsive to the 
t direction of look' of a person's eye. Eye movement is detected by constantly measuring the rate 
of change of potential difference between the electrodes and comparing it against a threshold rate 
of 2° per second. However, the "reading" limitations of the claims are not met, nor suggested 
and thus Jones cannot provide for the claimed features. 

A close reading of the Jones reference clearly shows that signal rate of change is 
compared flp«™*t a thr eshold value of eve movement measured in degrees/sec (see page 2, lines 
48-52 and lines 61-65 of the Jones reference) and not against a n accumulated numerical 
threshold value of reading (as required by claims L 12. 23. 24. and 25) . The examiner has 
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simply identified the term "threshold" absent a corresponding function and use. Terminology 
cannot be used in a vacuum simply because it is similar. First, Jones does not provide "any" 
leaching of "accumulating a numerical evidence". Jones only recognizes discrete instances when 
a threshold is crossed. In feet, until an eye movement threshold is crossed "no output" (26) is 
provided (page 2, line 64). Thus Jones explicitly cites tha t thev are not accumulating numerical 
evidence before the threshold is crossed and therefore cannot provide for the claimed feature. 
Second, Jones does not discuss any "reading" function tied to this threshold (only eye 
movement), and thus it is not seen how Jones can provide for accumulated numerical evidence of 
a parameter it is not monitoring. Jones is simply describing a mathematical way of measuring 
eye movement, and thus one cannot attribute claimed features (such as recognizing reading, 
skimming, and scanning) not discussed by Jones. 

Applicants thus contend that the combination of Tognazzmi and Jones does not make the 
claimed invention obvious. The above-mentioned arguments equally apply to all independent 
claims (1,1 2, 23, 24, and 25). 

One specific example of the claimed invention (as outlined in page 9, line 10 to page 10, 
line 1 of the application as filed) provides for a reading-evidence variable that can be 
incremented by 1 when the eye moves to the right and decremented by -1 when the eye moves 
towards the left * The reading evidence is then compared against a predetermined threshold (say 
3) before reading is detected. This is in stark contrast with the method of Tognazzmi wherein a 
region of interest is detected (not actual reading) based upon the amount of time spent gazing at 
that particular region in a computer screen. 

As to independent claim 24, the examiner has rnischaracterized the claim. The examiner 
cites on page 4, lines 8-1 1, that "claims 24 and 25, only differs from claim 12 and 23 in that the 
term "computer program instructions** is used in the claims..." However, a closer reading of 
claim 24 shows that it includes features "averaging over a predetermined period of time" (lines 5 
and 6), and "switching modes from a scanning mode and a slamming mode to a reading mode 
when reading is detected" (lines 1 1 and 12). Absent a rejection of these elements, the examiner 
has railed to establish a prima facia case of obviousness as is required. Thus, the rejection of this 
claim is incomplete and without merit 

Regarding dependent claims 4-5, Tognazzmi fails to provide for a threshold-based 
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reading detection method that includes a step for switching between a swnning/skimiiiing mode 
to a reading mode (when reading is detected upon reaching the threshold). As mentioned above, 
the Jones reference clearly shows that signal rate of change is compared against a threshold value ' 
of eye movement and not against a numerical threshold value of reading. Thus, no switching 
function is implied or suggested by the combination of Tognazzini and Jones. 

With regard to dependent claims 6 aid 10-11, applicants contend that Tognazzini 
provides for a method for monitoring eye gaze patterns to determine a level of interest Thus, the 
method of Tognazzini measures the time period and location (within the screen) of an eye gaze. 
Thus, if a user stares at a particular area of a computer screen (see column 10, lines 5-10 of 
Tognazzini), the method of Tognazzini identifies the article corresponding to that particular area 
and articles related to the identified articles are provided to the user. By sharp contrast, the 
present invention's method detects reading (as opposed to gazing or staring) via a threshold and 
in the instance that reading is detected, the present invention extracts the text (see "reading said 
heterogeneous content" aspect of claim 1) that was read and retrieves additional information 
(retrieved from a database over a network). 

Regarding independent claim 7 and 8, applicants contend that method of Tognazzini fails 
to provide for an accurate model of users as provided by the claimed invention. The method of 
Tognazzini identifies what applications on a computer screen (or areas on a computer screen) are 
getting the user's attention and any such identified applications or areas are then used to fine tune 
additional information (e.g, advertisements) provided. On the other hand, the present 
invention's method first detects what information was read by a user (based upon a threshold), 
and upon detection of reading, the present invention's method extracts the read text and fine 
tunes accurate models and advertisements based upon the read text that was extracted. 

As to claim 9, applicants refer the Examiner to section D of page 12 of the application as 
filed. This specifically outlines the use of the niethod of the present invention to provide for 
accurate help in computer applications. For example, by analyzing read text, a system can 
provide accurate help based upon the read text (see context-sensitive help of section D of page 
12 of the application as filed). Tognazzini mils to disclose or suggest the accurate help feature of 
the present invention. 
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The above-mentioned arguments for dependent claims 2-11 equally apply to dependent 



As has been detailed above, none of the references, cited or applied, provide for the 
specific claimed details of applicant's presently claimed invention, nor render them obvious. It is 
believed that this case is in condition for allowance and reconsideration thereof and early 
issuance is respectfully requested. ( 

As this Appeal Brief has been timely filed within the set period of response, no petition 
for extension of time or associated fee is required. However, the Commissioner is hereby 
authorized to charge any deficiencies in the fees provided, to include an extension of time, to 
Deposit Account No. 09-0441. 



claims sets 14-22. 



Summary: 




Randy W. Lacasse 
Reg. No. 34,368 



1725 Duke Street 
Suite 650 

Alexandria, VA 22314 
(703)838-7683 
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Appendix: 

1 . A method for recognizing reading, skimming, and scanning modes from eye-gaze 
patterns, said method comprising the steps of: 

quantizing eye movements of a user viewing heterogeneous content in both X and Y 
axes; accumulating a numerical evidence of reading until a predetermined threshold is reached; 
and 

detecting reading when said numerical evidence of reading exceeds said threshold. 

2. A method for recognizing reading, skimming, and scanning modes from eye-gaze 
patterns, as per claim 1 , wherein said quantizing step includes averaging over a predetermined 
period of time. 

3. A method for recognizing reading, skimming, and scanning modes from eye-gaze 
patterns, as per claim 1, wherein said heterogeneous content includes at least one of the 
following: text, images, hyperlinks, windows, icons, and menus. 

4. A method for recognizing reading, skimming, and scanning modes from eye-gaze 
patterns, as per claim 1, comprising the further step of switching modes from a skimming mode 
and a scanning mode to a reading mode when said reading is detected. 

5. A method for recognizing reading, skimming, and scanning modes from eye-gaze 
patterns, as per claim 4, comprising the further step of adapting to said user's need if said reading 
is detected. 

6. A method for recognizing reading, skimming, and scanning modes from eye-gaze 
patterns, as per claim 5, wherein said adapting step comprises the further steps of: 

recording -said heterogeneous content that is of interest to said user, 
finding relevant information from a database using said recorded heterogeneous content; 
and utilizing said found relevant information to adapt to a user's needs. 
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7. A method for recognizing wading, skimming, and scanning modes from eye-gaze 
patterns, as per claim 6, wherein said utilizing step comprises the further step of developing 
accurate models of said users. 

8. A method for recognizing reading, skimming, and scanning modes from eye-gaze 
patterns, as per claim 6, wherein said utilizing step comprises the further step of deterrninmg 
fine- grained information regarding said user's interest in Internet advertising. 

9. ' A method for recognizing reading, skimming, and scanning modes from eye-gaze 
patterns, as per claim 6, wherein said utilizing step comprises the further step of providing more 
accurate help regarding computer applications. 

10. A method for recognizing reading, skimming, and scorning modes from eye-gaze 
patterns, as per claim 6, wherein said database includes at least one of the following: a local 
machine database, a local area network database, and a wide area network database. 

11. A method for recognizing reading, skimming, and scanning modes from eye-gaze 
patterns, as per claim 10, wherein said wide area network database is the world wide web. 

1 2. A system for recognizing reading, skimming, and scanning modes from eye-gaze 
patterns, said system comprising: 

an eye-movement quantizer that quantizes eye movements of a user viewing 
heterogeneous content in both X and Y axes; 

a reading-evidence accumulator that accumulates a numerical evidence of reading; 

a threshold-comparator that compares said numerical evidence of reading against a 
predetermined threshold; and 

a reading-detector that detects reading when said numerical evidence of reading exceeds 
said predetermined threshold. 
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13. A system for recognizing reading, skimming, and scanning modes from eye-gaze 
patterns, as per claim 12, wherein said eye-movement quantizer averages over a predetermined 
period of time. 

14. A system for recognizing reading, skimming, and scanning modes from eye-gaze 
patterns, as per claim 12, wherein said heterogeneous content includes at least one of the 
following: text, images, hyperlinks, windows, icons, and menus. 

1 5. A system for recognizing reading, skimming, and scanning modes from eye-gaze 
patterns, as per claim 12, further comprising a mode-switcher that switches modes from a 
skimming mode and a scanning mode to a reading mode when said reading is detected. 

16. A system for recognizing reading, skimming, and scanning modes from eye-gaze 
patterns, as per claim 1 5, wherein said mode-switcher generates a user adaptation command 
signal when switching to said reading mode. 

17. A system for recognizing reading, skimming, and scanning modes from eye-gaze 
patterns, as per claim 1 6, wherein, in response to said user adaptation command signal: 

records said heterogeneous content that is of interest to said user, 
finds relevant information from a database using said recorded heterogeneous content; 
and utilizes said found relevant information to adapt to a user's needs. 

18. A system for recognizing reading, skimming, and scanning modes from eye-gaze 
patterns, as per claim 17, wherein the system develops accurate models of said users. 

19. A system for recognizing reading, skimming, and scanning modes from eye-gaze 
patterns, as per claim 17, wherein the system determines fine-grained information regarding said 
user's interest in Internet advertising. 
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computer program instnictions for detecting reading when said numerical evidence of 
ceading exceeds said predetermined threshold; and 

computer program instructions for switching modes fioma scanning mode and a 
summing mode to a reading mode when reading is detected. 

25 . An article of manufecture comprising a computer program product having a machine- 
readable medium including computer program instructions embodied therein for utilizing user 
interest information to adapt a computer to a user's needs with: 

computer program instructions for recording eye-gaze patterns of said user viewing 

heterogeneous content; 

computer program instructions for determining from said recorded patterns whether said 

user is reading, skimming, and scanning; 

computer program instructions for recording heterogeneous content of interest to said 

user upon detection of said reading; 

computer program instructions for finding relevant information from a database using 
said recorded heterogeneous content of interest; and 

computer program instructions for adapting said computer to said user's needs using said 
relevant information. 
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